
Image Processing Apparatus, Image Processing Method, and Computer 
* Readable Recording Medium Recording Image Processing Program for 

Processing Image Obtained by Picking Up or Reading Original 

5 [000 1] This application is based on application No. 2000-26096 1 filed in 
Japan, the content of which is hereby incorporated by reference. 
BACKGROUND OF THE INVENTION 
Field of the Invention 

[0002] The present invention relates to apparatus and method for image 
10 processing, and a computer readable recording medium in which an image 
processing program is recorded. More particularly, the present invention 
^0 relates to an image processing apparatus, an image processing method, and 

Q a computer readable recording medium recording an image processing 

tJ^ program for processing an image obtained by picking up or reading an 

15 original. 
Hi Description of the Related Art 

]^ [0003] Conventionally, a non-contact type image pickup device ILke a 

ly digital camera and a contact type input device like an image scanner have 

been used to convert an original having an image formed on a paper or the 

20 hke to digital data. Once an original paper is folded, a fold hne or a fold 
may remain thereon. When shooting such an original including the fold 
with a digital camera or reading it with an image scanner, the amount of 
reflected light would differ in the folded portion from the remaining portion 
due to surface irregularity at the folded portion. As a result, the image 

25 obtained by shooting would include noise at the folded portion, with its 

luminance value varied from that of the remaining portion. Similarly, the 
image read by the image scanner would include noise at the folded portion. 
[0004] Fig. 13 shows an original to be shot by a digital camera. The 
original 200 includes a region 201 having a photograph therein and a region 

30 203 having text therein. Original 200 has been folded and has a fold line 
205 remained thereon. 

[0005] Fig. 14 schematically shows an image obtained by shooting the 
original shown in Fig. 13 by a digital camera. Referiing to Fig. 14, the 
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picked up image 250 includes a region 260 having original 200 expressed 
' therein. Region 260 in turn includes a region 261 corresponding to photo 

region 201 of original 200, a region 263 corresponding to text region 203, and 
shade 265 corresponding to a portion around fold line 205 of original 200. 
5 Shade 265 appears because of the difference in the amount of reflected light 
in the folded portion from the remaining portion of original 200 due to the ^ 
surface irregularity at fold line 205. 

[0006] When binarizing picked up image 205, an edge originally not 
existent on original 200 comes to appear, since the lightness is lower in 
10 shade 265 than in the remaining portion. This degrades accuracy in a 
^3 character recognition process or the like following the binarizing process. 

Si s 

,2 In the case where image 250 is subjected to a process for extracting a 

S rectangular region enclosing a photograph or the like therein, the accuracy 

O 

of extraction would also be degraded. Fig. 15 shows a photo region that 
iU 15 would be extracted when the photo region extracting process is conducted for 
["^ the image shown in Fig. 14. Referring to Fig. 15, the portion of shade 265 is 

O misjudged to have an attribute as a picture, so that the rectangular region 

267 has been extracted wrongly as the photo region. 
Q [0007] Besides the binarizing process, in a process of compressing an 

20 image picked up from an original graphic printed with a limited number of 
colors by indexing the colors, the rate of compression would be degraded due 
to a considerable increase in the number of colors at a portion within the 
image corresponding to the fold of the original. 

[0008] As a technique for correcting the shade appearing on an image 
25 obtained from pickup of an original due to a fold on the original, an image 

reader performing shading correction has been disclosed in Japanese Patent 
Laying-Open No. 1-165264. This image reader corrects a pixel value based 
on shading information obtained in advance from pickup of a white plate. 
[0009] Further, Japanese Patent Laying-Open No. 2-230870 discloses 
30 an image reader which detects a background luminance value of an original 
from an image obtained from pickup thereof, and eliminates the background 
based on the detected background luminance value. 

[0010] There is another method for correcting shading by additionally 
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providing a sensor to detect height information of an original and estimating 
urievenness in illuminance from the detected height of the original. A 
sensor may be of the type that obtains the height information from a 
radiated result of spot light or slit light. 
5 [00 1 1] These conventional techniques however exhibit various problems. 
In the image reader disclosed in Japanese Patent Laying-Open No. 1-165264, 
the white plate is used. Since the white plate does not include a fold at a 
position corresponding to that of the fold on the original, correction of such a 
fold cannot be made only with the shading information obtained from the 
10 white plate. 

p [0012] With the image reader disclosed in Japanese Patent Laying- 

^0 Open No. 2-230870, correction cannot be made for an image having almost 

^ no background region. 

O [0013] Further, in order to detect a local change in height of the original 

'A I 15 due to a fold or the like with the method utilizing the height information 
W obtained from a separate sensor, it is necessary to obtain the height 

%^ information from a large number of positions. This increases the number of 

pieces of the height information, thereby decreasing the processing speed. 

In addition, to detect a minute change in height, an expensive sensor will be 
p 20 required or the device size will be increased. There also arises a need for 

detailed information about lighting conditions including position, direction 

and intensity of indoor or solar light. 

SUMMARY OF THE INVENTION 

[0014] The present invention is made to solve the above-described 
25 problems. An object of the present invention is to provide an apparatus 

and a method for image processing allowing removal of a noise appearing on 
a Hghtness component within an image. 

[0015] Another object of the present invention is to provide an 
apparatus and a method for image processing allowing removal of a noise 
30 due to a fold on an original from an image. 

[00 16] A further object of the present invention is to provide a computer 
readable recording medium recording an image processing program to cause 
a computer to perform image processing allowing removal of a noise 
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appearing on a lightness component within an image. 

[0017] Yiet another object of the present invention is to provide a 

computer readable recording medium recording an image processing 

program to cause a computer to perform image processing allowing removal 

of a noise due to a fold on an original from an image. 

[0018] To achieve the above -described objects, the image processing 

apparatus according to an aspect of the present invention includes a 

receiving unit to receive an image, an edge detecting unit to detect an edge 

using a lightness component of the received image, a selecting unit to select 

a specific one of the detected edges, and a correcting unit to correct the 

lightness component of the selected specific edge, 

[0019] According to the present invention, it is possible to provide an 
image processing apparatus allowing removal of a noise appearing on the 
Ughtness component. 

[0020] Preferably, the image processing apparatus further includes a 
unit to extract an original included in the image, and the selecting unit . 
selects an edge continuously extending from a first end to a second end of the 
extracted original. 

[002 1] According to the present invention, the edge uninterrupted from 
the first end to the second end of the extracted original is selected. If the 
original includes a fold, the fold will appear on a picked up image as a noise 
of shade. It is often the case that such a fold of the original extends from an 
end to another end thereof. Thus, by selecting the edge continuously 
extending from the first end to the second end of the original included in the 
image, the noise resulting from the fold of the original can be selected from 
the image. As a result, it is possible to provide an image processing 
apparatus allowing removal of a noise due to a fold of the original from the 
image. 

[0022] According to another aspect of the present invention, the 
computer readable recording medium recording an image processing 
program records the image processing program to cause a computer to 
perform the step of receiving an image picked up from an original, the step 
of detecting an edge using a lightness component of the received image, the 



-4- 



1* 



step of selecting a specific one of the detected edges, and the step of 
correcting the lightness component of the selected specific edge. 
[0023] According to the present invention, it is possible to provide a 
computer readable recording medium recording an image processing 
program to cause a computer to perform image processing that enables 
removal of a noise appearing on the lightness component. 
[0024] Preferably, the selecting step of the image processing program 
includes the step of selecting an edge continuously extending from a first 
end to a second end of the original included in the image. * 
[0025] According to the present invention, it is possible to provide a 
computer readable recording medium recording an image processing 
program to cause a computer to perform image processing that enables 
removal of a noise due to a fold of the original from the image. 
[0026] According to another aspect of the present invention, the image 
processing method includes the step of receiving an image picked up from an 
original, the step of detecting an edge using a lightness component of the 
received image, the step of selecting a specific one of the detected edges, and 
the step of correcting the lightness component of the selected specific edge. 
[0027] According to the present invention, it is possible to provide an 
image processing method that enables removal of a noise appearing on the 
lightness component. 

[0028] Preferably, the selecting step of the image processing method 
includes the step of selecting an edge continuously extending from a first 
end to a second end of the original included in the image. 
[0029] According to the present invention, it is possible to provide an 
image processing method that enables removal of a noise resulting from a 
fold on the original from the image. 

[0030] According to yet another aspect of the present invention, the 
image processing apparatus includes an acquiring unit to acquire an image 
signal indicating an original image, an edge detecting unit to detect an edge 
in a lightness image primarily representing lightness of the original image 
as a Hghtness edge, a fold edge selecting unit to select any of the detected 
lightness edges that connects an edge corresponding to an end of an original 



to another edge corresponding to another end of the original as a fold edge 
attributable to a fold of the original, and a processing unit to process the 
image signal in the portion corresponding to the fold edge to eliminate an 
effect of the fold of the original on the original image. 
5 [003 1] According to the present invention, it is possible to provide an 

image processing apparatus allowing removal of a noise resulting from a fold 
of the original from the image. 

[0032] According to a further aspect of the present invention, the image 
processing apparatus includes an acquiring unit to acquire an image signal 
10 representing a color original image with three components, a color space 
converting unit to perform coordinate transformation of the image signal 
such that the color original image is represented by a hghtness component 
primarily representing lightness and another component, and a correcting 
O unit to correct the lightness component in a portion of the color original 

15 image that is detected as an edge portion in a lightness image including the 
y hghtness component and is undetected as an edge portion in a color 

difference image including the another component, so that the relevant 
portion is undetected as the edge portion in the lightness image. 



ry [0033] According to the present invention, the edge appearing only on 



g20 the hghtness component is selected as the noise. Thus, it is possible to 

provide an image processing apparatus allowing accurate removal of a noise 
of the Hghtness component from the image. 

[0034] According to a still further aspect of the present invention, the 
image processing apparatus includes an acquiring unit to acquire an image 

25 signal representing an original image, an edge detecting unit to detect an 
edge in a hghtness image primarily representing hghtness of the original 
image as a hghtness edge, and a correcting unit to correct a hghtness 
component in a portion of the original image detected as the hghtness edge 
when a difference in hghtness between portions of the original image each at 

30 a prescribed distance from the relevant hghtness edge on its respective sides 
is smaller than a prescribed threshold value, so that the relevant portion is 
undetected as the edge in the hghtness image. 

[0035] According to the present invention, a boundary of two regions 
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whose difference in lightness is at least a threshold value is prevented from 
being wrongly selected as the noise. Thus, it is possible to provide an image 
processing apparatus allowing accurate removal of a noise in the lightness 
component from the image. 
5 [0036] According to yet another aspect of the present invention, the 

image processing apparatus includes an acquiring unit to acquire an image 
signal indicating an original image, an edge detecting unit to detect an edge 
in a Ughtness image primarily representing lightness of the original image 
as a Ughtness edge, and a correcting unit to correct a lightness component in 
10 a portion of the original image detected as the hghtness edge when portions 
□ of the original image each at a prescribed distance from the relevant 

■S lightness edge on its respective sides have the same attribute of image, so 

? that the relevant portion is undetected as the edge in the lightness image. 

S [0037] According to the present invention, a boundary of two regions 

ns 15 having different attributes is prevented from being wrongly selected as the 
Id noise. Thus, it is possible to provide an image processing apparatus 

L allowing accurate removal of a noise of the hghtness component from the 

m image. 

[0038] The foregoing and other objects, features, aspects and 
Q 20 advantages of the present invention will become more apparent from the 
following detailed description of the present invention when taken in 
conjunction with the accompanying drawings. 
BRIEF DESCRIPTION OF THE DRAWINGS 

[0039] Fig. 1 shows a digital camera in accordance with an embodiment 
25 of the present invention, which is picking up an image from an original. 

[0040] Fig. 2 is a perspective view of the digital camera in accordance 
with the embodiment seen from its front. 

[0041] Fig. 3 is a block diagram showing a circuit configuration of the 
digital camera in accordance with the embodiment. 
30 [0042] Fig. 4 illustrates how a portion of an image corresponding to a 
fold of an original is detected. 

[0043] Fig. 5 shows pixel values on a line (X = XO) in a lightness image. 
[0044] Fig. 6 illustrates correction carried out by a fold erasing unit of 
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the digital camera in accordance with the embodiment. 

[0045] Fig. 7 shows pixel values on the line (X = XO) in the lightness 

image after the correction. 

[0046] Fig. 8 is a flow chart illustrating a fold correcting process 
5 conducted by the digital camera in accordance with the embodiment. 
[0047] Fig. 9 is a flow chart illustrating a fold detecting process 
conducted by the digital camera in accordance with the embodiment. 
[0048] Fig. 10 shows an example of the image picked up by the digital 
camera in accordance with the embodiment. 
10 [0049] Fig. 11 shows pixel values on a hne (X = XO) in the image shown 
in Fig. 10. 

=0 [0050] Fig. 12 shows pixel values on a line intersecting the detected 

'2 edge at right angles. 

O [0051] Fig. 13 schematically shows an original to be shot by a digital 

Y^l 15 camera. 

y [0052] Fig. 14 schematically shows an image picked up by a digital 

^ camera from the original shown in Fig. 13. 

[0053] Fig. 15 shows a photo region extracted from the image shown in 

Fig. 14 as a result of a photo region extracting process, 
g 20 DESCRIPTION OF THE PREFERRED EMBODIMENTS 
1=^ [0054] Hereinafter, an image processing apparatus in accordance with 

an embodiment of the present invention will be described, taking a digital 

camera as an example. In the drawings, the same reference character 

denotes the same or corresponding portion, and description thereof will not 
25 be repeated where appropriate. 

[0055] Referring to Figs. 1 and 2, digital camera 1 includes a shooting 

button 3, a pickup lens unit 4 and a card slot 5. 

[0056] The image shot by digital camera 1 is stored as electronic data on 
a hard disc card (not shown) placed within digital camera 1. Here, the hard 
30 disc card is a recording medium of image data, for which the one in 

conformity with PCMCIA (Personal Computer Memory Card International 
Association) may be used, or a memory card or the like may be used. 
Alternatively, a mini disc (MD) or a digital video disc (DVD) may be 
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employed as the recording medium. Further, it is also possible to output 
the image data directly to a printer or the like via a SCSI (small computer 
system interface) cable, for example, instead of recording it on the recording 
medium. 

5 [0057] Digital camera 1 of the present embodiment can improve image 
quahty when an original in the paper form, such as handout at a conference, 
catalog, magazine or research report, is recorded or output as electronic 
data. 

[0058] Fig. 3 is a block diagram showing a circuit configuration of 
10 digital camera 1 of the present embodiment. Referring to Fig. 3, digital 
p camera 1 includes a central processing unit (hereinafter, "CPU") 100 

^0 performing an overall control of digital camera 1, a charge coupled device 

% (hereinafter, "CCD") 104 performing an image pickup, a display unit 106 

O displa5dng an picked up image, a random access memory (hereinafter, 

15 "RAM") 108 temporarily storing the image from CCD 104, a color space 
\m converting unit 110 converting the picked up image to a prescribed color ^ 

space, a pre-treatment unit 112 performing a prescribed pre-treatment on 
m the converted image, a fold detecting unit 1 14 detecting a portion 

=^ corresponding to a fold in the original from the image having undergone the 

p 20 pre-treatment at pre-treatment unit 112, a fold erasing unit 116 removing 
1^ the detected fold, an output unit 1 18 outputting the image having the fold 

removed therefrom, a card memory unit 120 storing the image, and a read 

only memory (hereinafter, "ROM") 102 storing a program to be executed by 

CPU 100. 

25 [0059] Here, a reading device 122 may be connected to digital camera 1 
such that a program for the control of digital camera 1 can be read from 
CD-ROM 124, magneto-optical disc, digital video disc, flexible disk or the 
like. In this case, a program for causing CPU 100 to execute a fold 
correcting process, which will be described later, is recorded on a recording 

30 medium like CD-ROM 124 and is read by reading device 122, so that the fold 
correcting process is carried out. 

[0060] This fold correcting process may be carried out inside digital 
camera 1, or alternatively, it may be performed at another terminal such as 
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a personal computer or another camera connected to digital camera 1 by 
transferring the image data thereto. 

[0061] An actual shooting process will now be described with reference 
to the block diagram shown in Fig, 3. In Fig. 3, thick arrows represent the 
flow of the image, and thin arrows represent the flow of the control data. 
When a user turns on the camera, a scene being taken by pickup lens unit 4 
is displayed on display unit 106 via CCD 104. 

[0062] Upon detection of pressing of shooting button 3, CPU 100 
designates integration to CCD 104. When the integration is completed, it 
dumps the CCD data to RAM 108, and causes display unit 106 to display the 
image in freeze-frame. 

[0063] The image from CCD 104 is stored in RAM 108. Here, the 
image stored in RAM 108 is a fuU-color image consisting of multi-valued 
pixels of R (red), G (green) and B (blue) with 8 bits each. Once the image 
from CCD 104 is stored in RAM 108, it is subjected to respective processes in 
color space converting unit 110, pre-treatment unit 112, fold detecting unit 
1 14 and fold erasing unit 1 16. The image having thus undergone the fold 
correcting process is output from output unit 118 to card memory unit 120. 
[0064] This card memory unit 120 may be a hard disc, or may be an 
external storage device or terminal. 

[0065] Color space converting unit 110 converts the image stored in 
RAM 108 into color space having a lightness component. As the color space, 
YUV, L*a*b*, HSL or other color systems may be employed. In an 
environment for shooting an original, if the Ught radiated from a Hght 
source has a frequency component deflected to yellow or blue, for example, a 
new color system wherein a pixel value obtsuned by receiving the light 
radiated from the light source and reflected on a white plate is used as the 
Hghtness component without alteration may be established. 
[0066] Generally, a fluorescent light is used as the Hght source in the 
environment for shooting an original including text or the like. When the 
hght source is the fluorescent hght, it is considered that the component 
representing intensity of the Ught radiated from the fluorescent light is 
approximately equal to the Y component of the YUV color system. Thus, in 
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the present embodiment, color space converting unit 110 performs 
conversion to the color space of the YUV color system. 

[0067] Of the images converted by color space converting unit 110 into 
those expressed with lightness and other components, the one expressed 
only with the lightness component is called a lightness image, and the one 
expressed with the other components is c£Jled a color diEPerence image. 
[0068] Pre-treatment unit 112 performs pre-treatment for edge 
detection on the lightness image generated by color space converting unit 
110. As the pre-treatment, sharpening process, contrast emphasizing 
process, minimum value filtering process, maximum value filtering process, 
smoothing process for noise removal, and image downsizing process are 
conducted in this order. The sharpening process is performed to emphasize 
the edge of the image. The minimum value filtering process is performed to 
convert a pixel value within a prescribed block to a minimum value for the 
purpose of connecting the edge. The maximum value filtering process is 
conducted to replace a pixel value in a prescribed block with a maximum 
value so as to thin an edge that has become too thick or to remove a noise. 
The image downsizing process is conducted to reduce the amount of data 
used for noise removal and edge detection. 

[0069] It is preferred to change the order of these processes, the kinds or 
degrees of the processes as necessary, according to the entire configuration 
of the digital camera, Hghting conditions at the time of shooting, kinds of 
subjects or the like, to increase the accuracy of the edge extraction. 
[0070] Pre-treatment unit 112 may conduct the pre-treatment for the 
image before being converted by color space converting unit 110. 
[0071] Fold detecting unit 114 detects a portion of the image having 
undergone the pre-treatment corresponding to the fold of the original. To 
this end, an absolute value of quadratic differential is first calculated for the 
image having undergone the pre-treatment to generate an edge image. The 
edge image thus represents the degrees of changes of pixel values. 
Accordingly, an edge in an image corresponds to a portion within the image 
where the pixel values show sudden changes. The edges corresponding to 
ends of the original are then detected from the generated edge image. 



When the image includes the original therein, the edges corresponding to 
the ends of the original will be detected as an approximately rectangular 
shape. An edge connecting two points on these edges corresponding to the 
ends of the original is then detected as the edge corresponding to the fold of 
the original. This edge detecting process will be described later in detail. 
[0072] The edge detection may be conducted using any conventional 
techniques including those based on Hough transformation or edge tracing. 
[0073] To extract a region corresponding to the original, a method of 
detecting background luminance values from the image and using the 
detected results for the extraction, or a method utilizing edges may be 
employed. 

[0074] Fold erasing unit 1 16 conducts correction of the image stored in 
RAM 108 in its portion corresponding to the fold detected by fold detecting 
unit 1 14. This correction is performed only for the lightness image. In 
this correction, a pixel value of the lightness image is corrected according to 
its distance from a point on the portion corresponding to the detected fold. 
[0075] Fig. 4 illustrates how a portion of the lightness image 
corresponding to the fold of the original is extracted. The portion of the 
lightness image corresponding to the original 150 includes a plurality of 
edges. Among them, the one extending from an end to another end of 
original 150 is most likely to correspond to the fold of the original. In the 
example shown in Fig. 4, edges 151 and 152 are detected, since they are 
considered to correspond to the folds. Other edges are not detected as those 
corresponding to the folds, since they are considered as edges included in 
pictures or letters within the original. 

[0076] Fig. 5 shows pixel values on a line X = XO in the lightness image 
shown in Fig. 4. Referring to Fig. 5, there is a sudden change of the pixel 
values at a position PO. Thus, the position PO is detected as an edge. 
[0077] Fig. 6 illustrates a correcting method employed at the fold 
erasing unit of the digital camera of the present embodiment. In Fig. 6, the 
horizontal axis represents the position X of a pixel, and the vertical axis 
represents the pixel value in the Hghtness image. Referring to Fig. 6, a 
maximum correction amount V is determined as half the amount of change 
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in pixel value between positions Q and R that are each at a minute distance 
d from the position P of the detected edge. The correction of the lightness 
image is performed for a region from a position Xb to a position Xa, where 
position Xb is a distance D apart from edge position P in the positive 
5 direction, and position Xa is distance D apart from edge position P in the 
negative direction. A pixel value Yl at a position XI in the negative 
direction from edge position P is set to Y1 + V x (XI - Xa) / D. A pixel value 
Y2 at a position X2 in the positive direction from edge position P is set to Y2 
- V X (Xb - X2) / D. Thus, the pixel values around edge position P in the 
10 Hghtness image are corrected and the change at position P is smoothed. 
[0078] Besides this correcting method, any other method may be 
employed which can smooth out the changes of the pixel values in the 
lightness image around edge position P. 

[0079] Fig. 7 shows pixel values on the line X = XO in the hghtness 

15 image after correction. Referring to Fig. 7, the pixel values around edge 

position PO have been corrected to smooth out the change at the position PO. 
[0080] The fold correcting process being conducted by the digital camera 
of the present embodiment will now be described. Fig. 8 is a flow chart 
showing the flow of the fold correcting process conducted by digital camera 1, 

20 Referring to Fig. 8, in the fold correcting process, color space converting unit 
110 first converts the image stored in RAM 108 into a lightness image and a 
color difference image in the color space of YUV color system (step Si). 
Thus, the hghtness image composed only of the lightness component can be 
obtained from the fxxU-color image shot by the camera. The following 

25 processes are conducted for the color space of the YUV color system. 

[0081] Pre-treatment unit 1 12 conducts prescribed pre-treatment on the 
hghtness image. As the pre-treatment, sharpening process, contrast 
emphasizing process, maximum value filtering process, minimum value 
filtering process, downsizing process and smoothing process are conducted 

30 in this order (step S2). 

[0082] Fold detecting unit 1 14 detects a portion of the hghtness image 
corresponding to the fold of the original (step S3). A correction of the 
hghtness image is then made for the detected portion corresponding to the 
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fold (step S 4), and the process is terminated. 

[0083] Fig. 9 is a flow chart showing the flow of the fold detecting 
process conducted by digital camera 1 of the present embodiment. The fold 
detecting process corresponds to step S3 of Fig. 8. Referring to Fig. 9, in the 
fold detecting process, an edge image is first generated by calculating an 
absolute value of quadratic differential for each pixel of the lightness image 
having undergone the pre-treatment (step Sll). In the edge image, 
portions where the pixel values of the Ughtness image exhibit sudden 
changes are extracted as edges, which are differentiated from portions 
where the pixel values of the lightness image do not make sudden changes. 
[0084] Using the extracted edges, a region corresponding to the original 
included within the image, or the original region, is extracted (step Sl2). 
The extraction of this original region is performed by extracting edges 
forming a rectangle. Alternatively, the original region may be extracted by 
utilizing detected results of background luminance values. 
[0085] After the extraction of the original region, the edges included 
therein are examined, and any one continuously extending from an end to 
another end of the original is extracted therefrom (step Sl3). Specifically, 
the edges located in the vicinity of the ends of the original are first detected. 
It is then determined, for each pair of any two of the detected edges, whether 
there is an edge extending in the same direction and connecting the relevant 
pair of edges. In other words, it is determined whether there is an edge on 
a hne connecting the arbitrary two edges. If so, the relevant edge is 
extracted as the edge extending continuously from an end to another end of 
the original. 

[0086] If there is no edge connecting the arbitrary two edges, it is 
determined that the relevant pair of edges does not satisfy the requirement. 
Repeating this examination for every pair of edges located in the vicinity of 
the ends of the original, all the edges extending continuously from an end to 
another end of the original are detected. 

[0087] As the edge detecting method, the conventional methods based 

on the Hough transformation or edge tracing may be employed. 

[0088] In the next step Sl4, it is determined whether each detected edge 
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continuously extending from an end to another end of the original truly 
corresponds to the fold of the original. For this confirmation, a method ., 
employing a color difference image, a method utilizing a background region 
of the image, and a method utilizing an attribute of the image may be 
employed by selectively using one of them or by combining two of them. 
[0089] 

[0090] Method Employing Color Difference Image 
[0091] In this method, it is examined whether the edge continuously 
extending from an end to another end of the original extracted from the 
Hghtness image is also detected from the color difference image. If not, it is 
determined that the relevant edge corresponds to the fold of the original. 
This is because the noise due to the fold of the original would not change the 
color on the original, so that it shovild appear only on the Hghtness image. 
[0092] Method Utihzing Lightness of Background Region 
[0093] This method is advantageous for an original in which 
background regions different in lightness are located adjacent to each other. 
Fig. 10 shows an example of the image shot by digital camera 1 of the 
present embodiment. Referring to Fig. 10, the original included in the 
image has a title region and another region that are different in background 
color from each other. 

[0094] With this kind of original, a boundary between the title region 
and the other region will be extracted as the edge continuously extending 
from an end to another end of the original. However, if the correction as 
above is made for this edge, such an unnecessary correction will deteriorate 
the image quahty. Thus, according to the present invention, background 
pixel values on opposite sides of the edge connecting an end to another end 
of the original are taken into consideration. In this example, they 
correspond to the background pixel values in the title region and in the other 
region each at a prescribed distance from the boundary. 
[0095] Fig. 1 1 shows pixel values on the line X = XO within the hghtness 
image shown in Fig. 10. Referring to Fig, 11, the background pixel values 
at positions at respective distance Yl, Y2 in opposite directions from the 
position P of the edge connecting two ends of the original are compared with 
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each other. In this example, the background pixel values in the title region 
and in the other region are different from each other. In such a case, it is 
determined that the extracted edge continuously connecting the two ends of 
the original does not correspond to the fold of the original. 
[0096] Fig. 12 shows pixel values on a line perpendicular to another 
detected edge. In the case where the background pixel values at positions 
at respective distance Yl, Y2 in opposite directions from the edge position P 
are equal to each other, it is determined that the relevant edge corresponds 
to the fold of the original. Thus, in this case, the correction is made for the 
relevant portion corresponding to the fold in step S4 of Fig. 8. 
[0097] Method Utilizing Attribute of Image 
[0098] In the method utilizing an attribute of an image, the 
determination as to whether the extracted edge truly corresponds to the fold 
of the original is made using the image attribute, instead of the lightness of 
the background regions used in the above-described method. Here, the 
image attribute of a region having a photograph included therein becomes a 
picture attribute, while a region having text therein exhibits a text 
attribute. 

[0099] In this method, in the case where two regions at the same 
distance from a detected edge in opposite directions perpendicvdar to the 
edge have the same image attribute, it is determined that the relevant edge 
corresponds to the fold of the original. If the two regions have the image 
attributes different from each other, it is determined that the relevant edge 
does not correspond to the fold of the original. Thus, in the case where a 
region including a photograph and exhibiting the picture attribute and a 
region including text and exhibiting the text attribute are located adjacent 
to each other, the edge of their boundary is prevented from being wrongly 
extracted as the edge corresponding to the fold of the original. As a resxilt, 
an unnecessary correction and hence degradation of the image quality can 
be suppressed. 

[0100] As explained above, according to the digital camera of the 
present embodiment, it is possible to remove a noise resulting from a fold of 
an original from an image picked up from the original. Since the correction 
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can be made using only the picked up image, it is unnecessary to 
additionally provide a reference plate for shading or a device for measuring 
a shape of the original. Thus, the configuration is simplified. 
[01011 Although the fold correcting process for the color image has been 
described representatively above, the digital camera of the present 
embodiment can also perform the similar process on a monochrome image. 
In this case, the process of determining continuity of the color difference 
component becomes unnecessary, since the monochrome image itself is the 
hghtness image. 

[0 1021 For the purposes of speeding the processing and simplifying the 
device configuration, it is also possible to eliminate the color space 
conversion even when using the color image. In this case, edges considered 
to correspond to the folds are extracted from respective plane images, from 
which any one appearing in all the three plane images is extracted as the 
edge corresponding to the fold. Further, only the G plane image of the color 
image can be used, instead of the lightness image, to conduct the process 
according to the present embodiment. 

[0 103] Although the fold correcting process of the present embodiment 
has been described to be performed within the digital camera, it is also 
possible to read the fold correcting program recorded on a recording medium 
in a personal computer for execution of the program therein. In this case, 
the image shot by the digital camera can be sent to the personal computer, 
where the noise corresponding to the fold can be removed from the shot 
image. 

[0 104] According to the digital camera of the present embodiment, an 
edge extending from an end to another end of the original is first selected, 
and it is then confirmed whether it corresponds to the fold of the original 
with "the method emplo5dng the color difference", "the method utilizing the 
background region of the image", or "the method utilizing the image 
attribute". Alternatively, it is also possible to confirm an edge 
corresponding to the fold of the original by first selecting edges with any of 
"the method employing the color difference", "the method utilizing the 
background region of the image", "the method utilizing the image attribute" 
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or a combination thereof, and then selecting among them the one extending 
from an end to another end of the original. For an image which does not 
include ends of the original therein, the edge corresponding to the fold of the 
original may be detected using any of "the method employing the color 
difference", "the method utilizing the background region of the image", "the 
method utilizing the image attribute", or a combination thereof. 
[0105] Further, for an image which includes an original under a limited 
condition, e.g., an original including only a text region and having its 
background color uniform through the original, it is possible to determine an 
edge extending from an end to another end of the original included within a 
generated edge image as the edge corresponding to the fold, without further 
processing. 

[0106] Although color space converting unit 106 has been provided in 
the present embodiment for converting the obtained image to the color space 
having the lightness component, there are cases where such a unit is 
unnecessary. An example thereof is the case where the image processing 
apparatus acquires as its input image an image represented by the color 
space having the lightness component. Another example is the case where 
R, G and B values of each pixel are integrated to obtain a lightness value of 
the relevant pixel, and a lightness image is formed of those lightness values. 
In this case, the R, G and B values can be increased or decreased in scale in 
the portion extracted as an edge corresponding to the fold, so that the 
correction to eliminate a noise resulting from the fold of the original is 
realized. 

[0107] Although the present invention has been described and 
illustrated in detail, it is clearly understood that the same is by way of 
illustration and example only and is not to be taken by way of limitation, the 
spirit and scope of the present invention being limited only by the terms of 
the appended claims. 
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